Lightweight Deep Neural Network Research Articles

Evaluation of strength and stiffness degradation of RC shear walls: An integrated image processing and deep learning approach

AbstractIn the aftermath of an earthquake, damage detection and performance evaluation of structural components are imperative for assessing the residual seismic capacity of a building. In this study, an integrated image processing and deep learning approach was developed to evaluate the degradation in strength and stiffness (i.e., strength reduction and stiffness reduction) of reinforced concrete (RC) shear walls. The approach comprised two main tasks: detecting and localizing visible seismic damage from photographs and evaluating strength and stiffness degradation based on this information. The semantic segmentation network, Damage‐Net, was used for damage detection and localization. A novel crack morphological processing layer and a patch feature extraction layer were developed for damage feature extraction and compression. A lightweight deep convolutional neural network named DegradeEval‐Net_v2, featuring the upgraded dilated and separable convolution block and multi‐layer perception, was developed to link the damage feature with strength and stiffness degradation. A database comprising test data and photographs of 14 RC shear wall specimens with a flexural‐dominated behavior mode and high to intermediate ductility was constructed to train and test the DegradeEval‐Net_v2 network. The results indicate that DegradeEval‐Net_v2 substantially improved the performance assessment accuracy of damaged RC shear walls, with a 35% smaller root mean square error (RMSE) for stiffness degradation evaluation and 75% smaller RMSE for strength degradation evaluation, compared with the provisions specified in JBDPA and FEMA guidelines. Moreover, evaluation results on test sets demonstrate that introducing the damage feature extraction and compression layers effectively preserved local crack information and improved the accuracy with which stiffness reduction was evaluated. In addition, DegradeEval‐Net_v2 outperformed ResNet18 and MobileNet V3 in terms of balanced efficiency and accuracy. Interpretability analysis demonstrates that the model learned the distinct contribution patterns of various visible damage indexes to stiffness and strength degradation across different loading levels.

Lightweight Deep Neural Network with Data Redundancy Removal and Regression for DOA Estimation in Sensor Array

Lightweight Deep Neural Network with Data Redundancy Removal and Regression for DOA Estimation in Sensor Array

Fine-grained food image classification and recipe extraction using a customised deep neural network and NLP

Global eating habits cause health issues leading people to mindful eating. This has directed attention to applying deep learning to food-related data. The proposed work develops a new framework integrating neural network and natural language processing for classification of food images and automated recipe extraction. It address the challenges of intra-class variability and inter-class similarity in food images that have received shallow attention in the literature. Firstly, a customised lightweight deep convolution neural network model, MResNet-50 for classifying food images is proposed. Secondly, automated ingredient processing and recipe extraction is done using natural language processing algorithms: Word2Vec and Transformers in conjunction. Thirdly, a representational semi-structured domain ontology is built to store the relationship between cuisine, food item, and ingredients. The accuracy of the proposed framework on the Food-101 and UECFOOD256 datasets is increased by 2.4% and 7.5%, respectively, outperforming existing models in literature such as DeepFood, CNN-Food, Wiser, and other pre-trained neural networks.

Explaining decisions of a light-weight deep neural network for real-time coronary artery disease classification in magnetic resonance imaging

In certain healthcare settings, such as emergency or critical care units, where quick and accurate real-time analysis and decision-making are required, the healthcare system can leverage the power of artificial intelligence (AI) models to support decision-making and prevent complications. This paper investigates the optimization of healthcare AI models based on time complexity, hyper-parameter tuning, and XAI for a classification task. The paper highlights the significance of a lightweight convolutional neural network (CNN) for analysing and classifying Magnetic Resonance Imaging (MRI) in real-time and is compared with CNN-RandomForest (CNN-RF). The role of hyper-parameter is also examined in finding optimal configurations that enhance the model’s performance while efficiently utilizing the limited computational resources. Finally, the benefits of incorporating the XAI technique (e.g. GradCAM and Layer-wise Relevance Propagation) in providing transparency and interpretable explanations of AI model predictions, fostering trust, and error/bias detection are explored. Our inference time on a MacBook laptop for 323 test images of size 100x100 is only 2.6 sec, which is merely 8 milliseconds per image while providing comparable classification accuracy with the ensemble model of CNN-RF classifiers. Using the proposed model, clinicians/cardiologists can achieve accurate and reliable results while ensuring patients’ safety and answering questions imposed by the General Data Protection Regulation (GDPR). The proposed investigative study will advance the understanding and acceptance of AI systems in connected healthcare settings.

Autonomous Identification of Bridge Concrete Cracks Using Unmanned Aircraft Images and Improved Lightweight Deep Convolutional Networks

The identification of the development of structural defects is an important part of bridge structure damage diagnosis, and cracks are considered the most typical and highly dangerous structural disease. However, existing deep learning-based methods are mostly aimed at the scene of concrete cracks, while they rarely focus on designing network architectures to improve the vision-based model performance from the perspective of unmanned aircraft system (UAS) inspection, which leads to a lack of specificity. Because of this, this study proposes a novel lightweight deep convolutional neural network-based crack pixel-level segmentation network for UAS-based inspection scenes. Firstly, the classical encoder-decoder architecture UNET is utilized as the base model for bridge structural crack identification, and the hourglass-shaped depthwise separable convolution is introduced to replace the traditional convolutional operation in the UNET model to reduce model parameters. Then, a kind of lightweight and efficient channel attention module is used to improve model feature fuzzy ability and segmentation accuracy. We conducted a series of experiments on bridge structural crack detection tasks by utilizing a long-span bridge as the research item. The experimental results show that the constructed method achieves an effective balance between reasoning accuracy and efficiency with the value of 97.62% precision, 97.23% recall, 97.42% accuracy, and 93.25% IOU on the bridge concrete crack datasets, which are significantly higher than those of other state-of-the-art baseline methods. It can be inferred that the application of hourglass-shaped depth-separable volumes can actively reduce basic model parameters. Moreover, the lightweight and efficient attention modules can achieve local cross-channel interaction without dimensionality reduction and improve the network segmentation performance.

Hybrid mix-up contrastive knowledge distillation

Knowledge distillation (KD) aims to build a lightweight deep neural network model under the guidance of a large-scale teacher model for model simplicity. Despite improved model efficiency through the KD technique, the performance gap between a teacher model and the trained student model remains significant. This is because the knowledge of the teacher model is not effectively transferred to the student model since the mapping landscape of the large-scale teacher model is not fully explored. To tackle this research gap, we propose a novel Hybrid Mix-up Contrastive Knowledge Distillation (HMCKD) approach, which facilitates a thorough and reliable mapping solution space exploration in order to significantly improve the performance of the student model. Specifically, we design a hybrid mixing strategy, including image-level mixing and feature-level mixing, to form a smoother mapping landscape as a means to provide a stronger guidance in order to embed its richer dark knowledge from a teacher model to its student model. Additionally, we apply two other strategies, including contrastive learning and top-k guided selection, in order to ensure more effective knowledge transferability from the teacher model. Extensive experiments have proved that our proposed HMCKD approach outperforms state-of-the-art knowledge distillation methods when tested on 6 publicly available datasets, such as CIFAR-100, CIFAR-100-C, STL-10, SVHN, TinyImageNet, and ImageNet. Particularly on CIFAR-100 dataset, the average accuracy of students using HMCKD increased by 1.47%. Further, both the visualization results and similarity quantifications have confirmed the narrowed knowledge gap between the teacher and student models. Our source code is available at https://github.com/lambett/HMCKD.

A TinyML solution for an IoT-based communication device for hearing impaired

Research on automatic translation of sign language to verbal languages has been progressively explored in recent years to assist speech and hearing-impaired people in communicating with non-signers. In this paper, a tiny machine learning (TinyML) solution is proposed for sign language recognition using a low-cost, wearable, internet-of things (IoT) device. A lightweight deep neural network is deployed on the edge device to interpret isolated signs from the Indian sign language using the time-series data collected from the motion sensors of the device. The scarcity of labeled training data is addressed by employing the deep transfer learning approach. Here, the knowledge gained from the data collected using the motion sensors of a different device is used to initialize the model parameters. The performance of the model is assessed in terms of classification accuracy and prediction time for different sampling rates and transferring schemes. The model achieves an average accuracy of 87.18% when all the parameters are retrained with just 4 observations of each sign recorded from the motion sensors of the proposed IoT device. The recognized sign is transmitted to a cloud platform in real-time. A mobile application, SignTalk, is also developed, which wirelessly receives the predicted signs from the cloud and displays it as text. Additionally, text-to-speech conversion is also provided on SignTalk to vocalize the predicted sign for better communication.

Noise Reduction of Nuclear Magnetic Resonance Spectroscopy Using Lightweight Deep Neural Network

Noise Reduction of Nuclear Magnetic Resonance Spectroscopy Using Lightweight Deep Neural Network

Intelligent fault diagnosis of multi-sensor rolling bearings based on variational mode extraction and a lightweight deep neural network

Intelligent fault diagnosis of multi-sensor rolling bearings based on variational mode extraction and a lightweight deep neural network

Non-sample fuzzy based convolutional neural network model for noise artifact in biomedical images

The use of a light-weight deep learning Convolutional Neural Network (CNN) augmented with the power of Fuzzy Non-Sample Shearlet Transformation (FNSST) has successfully solved the problem of reducing noise and artifacts in Low-Dose Computed Tomography (LDCT) pictures. Both the Normal-Dose Computed Tomography (NDCT) and the Low-Dose Computed Tomography (LDCT) images from the dataset are subjected to the FNSST decomposition procedure during the training phase, producing high-frequency sub-images that act as input for the CNN. The CNN creates a meaningful connection between the high-frequency sub-images from LDCT and their corresponding residual sub-images during the training operation. The CNN is given the capacity to distinguish between LDCT high-frequency sub-images and expected high-frequency sub-images, which frequently have varying levels of noise or artifacts, especially in a fuzzy setting. The FNSST-CNN then successfully distinguishes LDCT high-frequency sub-images from the expected high-frequency sub-images during the testing phase, thereby reducing noise and artifacts. When compared to other approaches like KSVD, BM3D, and conventional image domain CNNs, the performance of FNSST-CNN is impressive as shown by better peak signal-to-noise ratios, stronger structural similarity, and a closer likeness to NDCT pictures.

Lightweight Deep Neural Network Model With Padding-Free Downsampling

Lightweight Deep Neural Network Model With Padding-Free Downsampling

Deep reinforcement learning with light-weight vision model for sequential robotic object sorting

Training an agent to generate cooperative joint learning policies for executing object sorting in cluttered scenes is complex. This paper presents a comparative model-free deep reinforcement learning (DRL) system built from variants of end-to-end lightweight deep neural networks that jointly complement three primitive policies (pushing, grasping, and placing) that operate from visual observations to actions. The end goal of the proposed DRL systems is to perform object sorting in very dynamic and complex environmental scenarios of cluttered and random objects (irregular and regular blocks) existing in different color variations. For this, we investigated DRL methods built from 12 custom instances of Pixel-wise Q-valued Critic Networks (PQCN) from four main backbone networks (DenseNet121, DenseNet169, MobileNetV3, and SqueezeNet) individually combined with custom fully convolutional neural networks (FCN) for learning the image-based observation space through affordance mapping paradigm while considering both dual and single transfer learning schemes. Additionally, we explored three categories of gradient-based optimization methods and considered custom reward functions with varying discount factors. The results show that the PQCN constructed by integrating DenseNet121 or MobileNetV3 with FCN and trained by a dual transfer learning scheme (with a discount factor of γ=0.70) yielded the best performance in most categories of our evaluation in the training phases. Overall, the PQCN-DenseNet121, when trained with a dual transfer learning process, yielded the best generalization (sorting success rate) in most of the evaluation metrics across all object categories in the testing phase. Furthermore, our study provides a new benchmark for validating deep reinforcement learning algorithms while sorting object blocks under varying degrees of complexity. This research finds practical relevance in industrial fields such as manufacturing and packaging, building construction, plant sorting, automobile assembly lines, and waste sorting. Here is the article’s source code: https://github.com/Emmanuel-Okafor/DRL_for_Object_Sorting.git.

A lightweight deep neural network model and its applications based on channel pruning and group vector quantization

A lightweight deep neural network model and its applications based on channel pruning and group vector quantization

Lightweight Pixel Difference Networks for Efficient Visual Representation Learning.

Recently, there have been tremendous efforts in developing lightweight Deep Neural Networks (DNNs) with satisfactory accuracy, which can enable the ubiquitous deployment of DNNs in edge devices. The core challenge of developing compact and efficient DNNs lies in how to balance the competing goals of achieving high accuracy and high efficiency. In this paper we propose two novel types of convolutions, dubbed Pixel Difference Convolution (PDC) and Binary PDC (Bi-PDC) which enjoy the following benefits: capturing higher-order local differential information, being computationally efficient, and can be integrated well into existing DNNs. With PDC and Bi-PDC, we further present two lightweight deep networks named Pixel Difference Networks (PiDiNet) and Binary PiDiNet (Bi-PiDiNet) respectively to learn highly efficient yet more accurate representations for visual tasks including edge detection and object recognition. Extensive experiments on popular datasets (BSDS500, ImageNet, LFW, YTF, etc.) show that PiDiNet and Bi-PiDiNet achieve the best accuracy-efficiency trade-off. For edge detection, PiDiNet is the first network that can be trained without ImageNet, and can achieve the human-level performance on BSDS500 at 100 FPS and with [Formula: see text]1M parameters. For object recognition, among existing Binary DNNs, Bi-PiDiNet achieves the best accuracy and a nearly 2× reduction of computational cost on ResNet18.

Tiny-PPG: A lightweight deep neural network for real-time detection of motion artifacts in photoplethysmogram signals on edge devices

Photoplethysmogram (PPG) signals are easily contaminated by motion artifacts in real-world settings, despite their widespread use in Internet-of-Things (IoT) based wearable and smart health devices for cardiovascular health monitoring. This study proposed a lightweight deep neural network, called Tiny-PPG, for accurate and real-time PPG artifact segmentation on IoT edge devices. The model was trained and tested on a public dataset, PPG DaLiA, which featured complex artifacts with diverse lengths and morphologies during various daily activities of 15 subjects using a watch-type device (Empatica E4). The model structure, training method and loss function were specifically designed to balance detection accuracy and speed for real-time PPG artifact detection in resource-constrained embedded devices. To optimize the model size and capability in multi-scale feature representation, the model employed depth-wise separable convolution and atrous spatial pyramid pooling modules, respectively. Additionally, the contrastive loss was also utilized to further optimize the feature embeddings. With additional model pruning, Tiny-PPG achieved state-of-the-art detection accuracy of 87.4% while only having 19,726 model parameters (0.15 megabytes), and was successfully deployed on an STM32 embedded system for real-time PPG artifact detection. Therefore, this study provides an effective solution for resource-constraint IoT smart health devices in PPG artifact detection.

Defect detection of MicroLED with low distinction based on deep learning

MicroLED has great prospects in visible light communication, optical detection and other fields. To solve limited efficiency and reliability issues caused by current manual quality control processes in MicroLED production environments, we propose a micro-vision-based automatic scanning system (MVASS) used to capture the MicroLED defect images and performing defect prediction inference. Since MicroLED defects are weak, they are very similar to normal MicroLEDs with low distinction, and the large number of MicroLEDs in an image, making a poor detection effect for the existing method. In this paper, a real-time detection method named AMS-YOLO base on lightweight deep convolutional neural network for MicroLED defects with low-distinction is proposed, utilizing the attention residual module (ARM) to extract multi-level effective features by focusing on the key features in the defect images by self-attention, and introducing the multi-scale feature fast fusion module (M3FM) to fuses the multi-level feature effectively and quickly to better capture the global information of the defect image, and using the selective feature pyramid module (SFPM) to selectively fuse the extracted features to enhance the network's classification and positioning capabilities and ultimately achieve precise MicroLED classification and detection. Experimental results on the MicroLED dataset show that AMS-YOLO demonstrates a better performance, with the mAP, Precision, Recall and speed increasing by 3.0 %, 3.5 %, 6.0 % and 4.3 % respectively, compared to the YOLOv5s.

DualConv: Dual Convolutional Kernels for Lightweight Deep Neural Networks.

Convolutional neural network (CNN) architectures are generally heavy on memory and computational requirements which make them infeasible for embedded systems with limited hardware resources. We propose dual convolutional kernels (DualConv) for constructing lightweight deep neural networks. DualConv combines 3×3 and 1×1 convolutional kernels to process the same input feature map channels simultaneously and exploits the group convolution technique to efficiently arrange convolutional filters. DualConv can be employed in any CNN model such as VGG-16 and ResNet-50 for image classification, you only look once (YOLO) and R-CNN for object detection, or fully convolutional network (FCN) for semantic segmentation. In this work, we extensively test DualConv for classification since these network architectures form the backbone for many other tasks. We also test DualConv for image detection on YOLO-V3. Experimental results show that, combined with our structural innovations, DualConv significantly reduces the computational cost and number of parameters of deep neural networks while surprisingly achieving slightly higher accuracy than the original models in some cases. We use DualConv to further reduce the number of parameters of the lightweight MobileNetV2 by 54% with only 0.68% drop in accuracy on CIFAR-100 dataset. When the number of parameters is not an issue, DualConv increases the accuracy of MobileNetV1 by 4.11% on the same dataset. Furthermore, DualConv significantly improves the YOLO-V3 object detection speed and improves its accuracy by 4.4% on PASCAL visual object classes (VOC) dataset.

Smart-DNN+: A Memory-efficient Neural Networks Compression Framework for the Model Inference

Deep Neural Networks (DNNs) have achieved remarkable success in various real-world applications. However, running a Deep Neural Network (DNN) typically requires hundreds of megabytes of memory footprints, making it challenging to deploy on resource-constrained platforms such as mobile devices and IoT. Although mainstream DNNs compression techniques such as pruning, distillation, and quantization can reduce the memory overhead of model parameters during DNN inference, they suffer from three limitations: (i) low model compression ratio for the lightweight DNN structures with little redundancy, (ii) potential degradation in model inference accuracy, and (iii) inadequate memory compression ratio is attributable to ignoring the layering property of DNN inference. To address these issues, we propose a lightweight memory-efficient DNN inference framework called Smart-DNN+, which significantly reduces the memory costs of DNN inference without degrading the model quality. Specifically, ① Smart-DNN+ applies a layerwise binary-quantizer with a remapping mechanism to greatly reduce the model size by quantizing the typical floating-point DNN weights of 32-bit to the 1-bit signs layer by layer. To maintain model quality, ② Smart-DNN+ employs a bucket-encoder to keep the compressed quantization error by encoding the multiple similar floating-point residuals into the same integer bucket IDs. When running the compressed DNN in the user’s device, ③ Smart-DNN+ utilizes a partially decompressing strategy to greatly reduce the required memory overhead by first loading the compressed DNNs in memory and then dynamically decompressing the required materials for model inference layer by layer. Experimental results on popular DNNs and datasets demonstrate that Smart-DNN+ achieves lower 0.17%–0.92% memory costs at lower runtime overheads compared with the states of the art without degrading the inference accuracy. Moreover, Smart-DNN+ potentially reduces the inference runtime up to 2.04× that of conventional DNN inference workflow.

Large-scale Video Analytics with Cloud–Edge Collaborative Continuous Learning

Deep learning–based video analytics demands high network bandwidth to ferry the large volume of data when deployed on the cloud. When incorporated at the edge side, only lightweight deep neural network (DNN) models are affordable due to computational constraint. In this article, a cloud–edge collaborative architecture is proposed combining edge-based inference with cloud-assisted continuous learning. Lightweight DNN models are maintained at the edge servers and continuously retrained with a more comprehensive model on the cloud to achieve high video analytics performance while reducing the amount of data transmitted between edge servers and the cloud. The proposed design faces the challenge of constraints of both computation resources at the edge servers and network bandwidth of the edge–cloud links. An accuracy gradient-based resource allocation algorithm is proposed to allocate the limited computation and network resources across different video streams to achieve the maximum overall performance. A prototype system is implemented and experiment results demonstrate the effectiveness of our system with up to 28.6% absolute mean average precision gain compared with alternative designs.

Robust Payload Recognition Based on Sensor-Over-Muscle-Independence Deep Learning for the Control of Exoskeletons

Force myography (FMG) can detect changes in the muscle volume which can be interpreted to recognize human intention. FMG data, however, is highly dependent on the placement of sensors over muscles, and interpretation becomes challenging if the sensors get displaced. This paper presents a robust sensor over muscle independence (SOMI) preprocessing algorithm combined with lightweight deep neural network (DNN) which shows high classification accuracy of the FMG data. SOMI organizes the irregular sensory data into regular patterns. Proposed algorithm makes the DNN insensitive to not only position and rotation shift but also to the flip of the sensors arrangement. A custom designed FMG band is used for payload recognition to experimentally validate the proposed method with five payload statuses and eight subjects. A five-fold cross validation comparative study demonstrated that the proposed method is 19.8% and 7.1% more accurate than support vector machine (SVM) and DNN without SOMI, respectively, and showed superior performance against k-nearest neighbors (KNN) and decision tree (DT). SOMI empowered a lightweight DNN to maintain the accuracy over 98% for different arbitrary wearing schemes of the FMG band over both left and right upper arms.